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ABSTRACT 

A new instrument for classroom observation called 
the Personal Record of School Experience (PROSE) , has been developed 
at Educational Testing Service. While PROSE was invented for a 
specific project involving 2000 4**year-old ghetto .children, its 
general applicability has already become apparent. PROSE represents a 
primitive method of classroom observation since it does not provide 
an observer rating type of classroom record. PROSE strives to 
eliminate observer subjectivity by requiring only the objective 
recording of observable events. The record is also personal since, the 
observer only watches one child at a time and records all his 
activity. PROSE is thus similar to ecological studies, except that 
the child's record is coded in the classroom rather than from a later 
review of films and recordings taken in class. The in-class coding is 
made possible by the PROSE "language,” based on 11 word statements 
.that can be fed directly to the computer. Static conditions, such as 
subject matter, class organization, and instructional materials are 
also recorded. The 11 word statements used by the observer code all 
aspects of the particular students's current activity, including 
level of attention, physical activity, and manifest affect. (MH) 
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OScAR GOES TO NURSERY SCHOOL; 



A NEW TECHNIQUE FOR RECORDING PUPIL BEHAVIOR 

In the time made available to me today I would like to share with you 
some notions about observational methodology in the context of a new instru- 
ment we have developed at Educational Testing Service. This instrument was 
developed for a particular project, but it has already become clear that it 
is likely to be useful in other studies and for other purposes < 

I have never been able to discuss methodological issues comfortably in 
"tiie abstract— in fact, I find it almost impossible to think constructively 
about them unless they arise- in connection with a specific research problem,. 
So it is that, when I was invited to participate in this symposium and share 
with you scMie of the things we have learned about classroom observation, I 
found myself turning automatically to our most recent attempt to apply the 
"OScAR Technique" to a practical research problem. And so it is that today 
I will be describing scxne of the constraints and demands placed on us by the 
nature of the study and some of the devices— —the methodological straws, you 
might say<— that we clutched at in meeting them 

Ever since T first became aware of their existence, my own concept of 
the role of structured observations in research has steadily evolved toward 
the more primitive. Originally I shared a notion which still has wide 
currency— the notion that the function of the classroom observer is to gener- 
ate an evaluative Judgment or opinion, usually referred to as a rating , of 
the teacher of the class. There are, I believe, certain purposes to be 
served by sudi ratings, Whenever the opinion of the classroom visitor is 
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the desired end-product, a rating "based on such a visit provides that opinion 
in a convenient form. But vhen a measure of the characteristic being rated 
is needed, ratings are untrustworthy indeed. Evidence that ratings by dif- 
ferent observers correlate highly does not necessarily mean they are react- 
ing to the trait they are supposed to be rating. What is needed is evidence 
that they correlate with some other measure. Attempts to do this with 
teadier ratings have not been very successful (Medley & Mitzel, I963, 
pp. 257-258). 

For a number of years many of us have been working under the assiamption 
that if the purpose of classroom observations is to yield measurements of 
classroom behavior, rather than observers’ opinions about teachers or pupils, 
something different from ratings must be used. If by measurements of behavior 
we mean mimbers which reliably discriminate among the classroons observed 
according to the types and amounts of events that occur in them, our observers 
must record the events themselves rather than opinions derived from them. 

If classrooms differ significantly with respect to frequencies of behaviors 
in a certain category, then that category is said to define a behavior 
dimension and the observational technique is said to measure that dimension. 

Implied in this view is the assmption that the constructor of the 
technique by the act of defining the items or categories that detes'mine which 
behaviors are to be recorded makes an a priori definition of the dimensions of 
behavior to be measxired by the technique. The verbal category systems de- 
vised by Withall (19^9) and Flanders (Amidon & Flanders, 1963)> for example, 
were both based on definite hypotheses about what dimensions of teacher 
behavior are important, and the fact that teachers were found to 
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differ with respect to the categories supported the theories behind the 
hypotheses . 

In my own work in collaboration with Harold Mitzel (Medley, 1963), 
we sou^t to move to a level I have already referred to as more primitive 
than this. The items or categories we designed for our observers were 
not intended to identify important dimensions; nor did we require that 
each one should discriminate among classes — or be of sufficient psychologi- 
cal interest to constitute a meaningful dimension of behavior if it did. 

Our only requirement was that different observers be able to agree about the 
frequencies of events in each category. If the number and variety of cate- 
gories used was sufficient, it should be possible to combine categories 
(with or without varying weights) into super-categories which would dis- 
criminate teachers along psychologically or pedagogically relevant dimensions. 

A somewhat homely analogy may serve to clarify this point. Suppose 
I am sending children to the zoo with the purpose of finding out what kind 
of animals are housed there. If I used a rating approach I would teach my 
observers how to recognize lions, tigers, elephants, ostriches, and then 
ask them to report back to me which of them they saw. If I used the OScAR 
technique, I would give them a list of characteristics such as number of 
legs, fur or feathers, color, etc., and as': them to check which characteris- 
tics were possessed by each beast they saw. From each child’s list I 
would then infer which animal that child had seen. 

In the second approach my conclusions would be much less dependent for 
their accuracy on the children’s ability to recognize animals; indeed, my 
results might identify some species I had not expected them to see at all. 

The acciwacy of my conclusions would depend partly on the completeness of 
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my checklist of things to look for, and partly on the children's ability to 
count legs, discriminate fur from feathers, recognize colors, etc. If these 
discriminations were primitive enough, based on obvious cues, and if my list 
were sufficiently varied and complete, my conclusions should approach 100^ 
accuracy. 

Similarly, if I ask a classroom observer only to make discriminations 
based on simple overt behaviors, and if I can construct a sufficiently com- 
prehensive list of them for him to record, it becomes possible to secure 
descriptions of dimensions of behavior virtually independent of the training 
or experience of the observers . Indeed, there is no reason why an observer 

should even know what dimensions are being measured. This is the sense in 

* 

which I refer to the OScAE technique as more primitive than category sys- 
tems like those of Withall and of Flanders already cited. 

Before turning to the specific application I wish to discuss, let me 
emphasize the fact that any individual item on a schedule may contain veiy 
little information, but a combination of several occurring together can 
pinpoint an important piece of data. To know only that an animal has hoofs, 
or fxir, or stripes does not tell us much. To know that he has all three 
makes us pretty nearly certain that he is a zebra « It is by piling up 
a number of simple facts that we develop useful information. 

With this as background I would like to describe a new observation 
system we have designed, one in which we have tried to adapt these ideas 
to a new study. 

The study in question is a large-scale longitudinal study of early 
childhood education, A sample of 2,000 foui'-year-old children from ghetto 
areas will be identified, studied intensively for a five-year period 
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ending when most of them are in the third grade. Particular emphasis is to 
he placed upon the experiences each individual child has in school, in- 
cluding any nursery school or Head Start center he may attend. One purpose 
of the study is to identify characteristics of these early classroom ex- 
periences which are associated with success in school hy the time the 
child reaches the third grade. 

The requirements of the study I have mentioned set some important 
constraints on us. First, the sheer size of the project is a factor to 
he considered. The proposal is to secure a minimum of 20 observations 
per year of each of 2,000 children for five years. That means 40,000 
records in the first year, and (allowing for attrition) something like 
150,000 total. 

Second, we are being asked to observe pupils instead of teachers. 

Most of our past experience has been gained in observing teachers — a 
much easier task. For one thing, there is only one teacher in a class, as 
a rule — and his role is central. In that same classroom there are between 
15 and 45 pupils, whose roles vary from central to peripheral, and who do 
not always act as if they knew their roles. 

A third constraint arises from a decision to use as observers people 
recruited in the community, or at least in the subculture being studied. 
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The middle-class white observer has never been fully accepted in the ghetto — 
and in these times of social change is even less so. And even when he is, 
there is serious question whether he may not be so alien to the culture 
under study that his perceptions are distorted beyond the limits of useful- 
ness. This made it necessary to design an instrument which can be used by 
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observers who have no background either in psychology or pedagogy, and no 
understanding of or particular commitment to the scientific study of be- 
havior— perhaps, even with little or no formal education beyond high 
school. 

Fourth, a study like the one under discussion, designed to obtain 
comprehensive (if not exhaustive) data about a sample, requires something 
more like a descriptive record than a profile of measurements. Full 
exploitation of the potential of this study requires as comprehensive and 
many-faceted a record of the child’s experience as can be obtained. It 
must be possible to look at the data from many different angles, to test 
many different hypotheses — some of which, although today we do not suspect 
their existence, may seem of central importance five years from now. 

There were a number of other contra ints placed on the instrument to 
be constructed (some of which did not become apparent until they were 
encountered during the developmental process), but these were the major 
ones. Taken as a whole, they seemed to call for an instrument which would 
describe classroom , behavior as it related to the individual pupil, com- 
prehensively, objectively, and in a readily quantifiable form — and an 
instrument that a person completely inexperienced in structiired observations 
could learn to use easily and accurately. 

Let me turn now to the instrument we have developed, which we call the 
Personal Record of School Experience , or PROSE, and try to show how we have 
attempted to meet these requirements. 

The first thing to be noted is that the PROSE record is Tsersonal— the 
PROSE recorder focuses on one pupil at a time and records only what happens 
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to him. If the pupil attends to the teacher, for example, the teacher be- 
havior at that time is part of the child's experience and is recorded; 
if the pupil does not attend to the teacher, the teacher’s behavior at 
that time is not part of the child’s experience and is not recorded. 

If the behavior of each pupil in a class is sampled in this way, 
the behavior of the whole class has been sampled, and a description of 
the class as a whole may be constructed by combining individual records. 

But this process cannot be reversed. It is not possible to observe 
everything that happens in a class; and even if we could do so, we could 
not infer that all of it happened to the child we are interested in 

studying. 

As a matter of fact, one soon finds that it is not even practical to 
observe and record everything that happens to a single child. There is one 
group that claims to be able to do this— the ecological psychologists. 

Using the latest technological aids, the ecologist claims to be able to 
record everything that happens to a child, either by dictating a running 
narrative account into a tape recorder or by photographing the child’s be- 
havior on film or videotape. 

Whether an ecological report is complete or not, it is quite useless 
from a scientific point of view until it has been coded into quantitative 

form or, as I prefer to phrase it, until it has been translated into a 

language a computer can process. The task of coding such ^records is, for 
all pxactical purposes, the same task as the one that the classroom observ- 
er performs in the classroom. The coder working with the ecological 
record has one advantage over the classroom observer — the data he is 
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coding are recorded in permanent form, so that he can spend all the time 
he needs on the coding task. He can read and reread a transcript, view and 
review a film or videotape recording until he has it coded Just right. The 
classroom observer, on the other hand, must work in real time; he must code 
behaviors that appear once, then vanish forever. There is neither time to 
ponder a difficult decision nor opportunity to go back and reconsider, 

*’The moving finger writes, and having writ, moves on.” 

But the coder working from an ecological record must work with incom- 
plete data; the cues the camera or recorder may have missed can never be 
taken into account. On the other hand, the classroom observer has access 
to everything that happens (on camera or off), whether the ecological re- 
corder saw it as important enou^ to record or not. If behaviors are coded 
by our observers in the classroom, it takes one hour of coder time to code 
one ho\ir of behavior. If behaviors are coded from ecological records, 
it takes one hour to record one hour of behavior, and at least one more 
hour to code it. If any advantage is taken of the capability to rerun 
tapes or reread verbal records, it may take two or more hours to code one 
hour of behavior. Thus quantifiable behavior records based on ecological 
records may use anywhere from two to four or five times as many man-hours 
per hour of behavior as are needed when behavior is coded "live.” 

If an equal number of man-hours were invested in observing behavior and 
coding it "live" rather than in coding and recoding the same records, from 
two to five times as much data could be obtained with the same investment 
of time and money! Since our objective is to describe classroom behavior 
on the basis of observations of relatively small sample^ we must balance 
the larger observer errors made when behaviors are coded live against the 
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larger sampling errors incurred by ecological coders using a much smaller 
behavior sample. It is our experience that sampling errors tend to be much 
larger than the observer errors, so we have opted for in-class coders. 

The ecological record did, however, attract us as an ideal approach 
except for this one limitation — the inefficient use of time. ¥e decided, 
therefore, to attempt to design a technique by which the classroom observer 
might compile a record analogous to an ecological one but precoded, that is, 
an "ecological" record written in a language the computer could read , 
analyze, and interpret . 

Achieving this goal involved the accomplishment of three steps: (l) the 

development of a vastly simplified "language" — simplified both as to vocabu- 
lary and syntax j (2) the development of a way of writing the language so 
that the canputer could read it; and (3) the development of a procedure by 
which the classroom observer could record behaviors in that langiaage accu- 
rately and easily. 

The language we developed is based on 11-word statements made up 
by the observer to describe events observed in the classroom. Each word 
in a statement is a position in the statement which may be filled by one 
of two or more alternatives. The 11 words are represented on the front 
(or statement) side of the recording form (see Figure l) by numbers one 
to 11, each followed by the alternative choices for that word. Opposite 
each choice are 15 mark-sensing spaces, numbered to correspond to events 
numbered 1 to 5 in three cycles of five events each. 

The procedure may perhaps best be understood in terms of an example. 
Suppose that the event to be recorded is one in which the teacher is showing 
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the class how to plant a seed in a flowerpot, and that the pupil on whom the 
observation is focused is watching attentively^ The task of the observer, 
is, simply, to mark whichever alternative (if any) to each of the 11 words 
best describes this event. 

On the first word, for example, he marks "l.PART" in Box 1 (because 
this is event number one) to indicate that the child was a part of a group 
to which an adult was attending. On the second word he marks ’’2.TCHR" 

(again in Box l) to indicate that the adult in question was the teacher. 

On the third word he marks "3.SHTL" indicating that the teacher was show- 
ing or tellirg the pupils something. 

Words four and five are omitted as not applicable (they are used only 
for events involving peer interactions) and on the sixth word "6.MTL" is 
marked, to indicate that the event involved materials of some sort. On 
word seven the recorder indicates the sex and ethnic group mix in the event. ^ 
If, for instance, the pupil is of the same sex and ethnic group as the 
teacher, "7.SSSG" is marked. 

The eighth word is used to describe the level of attention the pupil 
is paying to the adult; in this case, conforming to the role expected of 
a pupil— i.e., paying attention— so "8. COOP" is marked. Word nine is used 
to indicate physical activity on the part of the pupil being observed. Since 
this pupil is not moving, word nine is left blank. The tenth word is 
used only when the pupil is alone, and the eleventh is used to indicate 
manifest affect; both are left blank in this example. 

Since events in the classroom occur rather rapidly, it is impossible 
for one observer to record all of the things that happen to a child during 
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the time he is under observation. Therefore, only a sample of such events 
is recorded, a sample which is determined hy a small timing device which 
the observer wears. This device emits a signal at fixed intervals (we 
are currently using 25 -second intervals) intp a hearing-aid type earphone 
in the observer’s ear. 

The observer locates the pupil to be observed, starts the timing device, 
and watches what happens to the pupil. When the timer sounds, he is to 
record a statement describing the event occurring at that moment. As soon 
as the statement has been recorded, the observer begins to watch the pupil 
again in preparation for the next signal. Five statements describing 
five events ccanplete one cycle . 

In addition to the 11-word statements used to describe transitory 
characteristics of each event, the language also provides means for de- 
scribing more stable conditions; this information, is recorded on the reverse 
or context side of the instrument (see Figure 2). 

Static conditions include the subject being taia^t, the way in which 
the class is organized, where in that organization the pupil fits, the 
general climate of the class, materials used by the child, and his location 
in the classrooms. Since all of these things are relatively stable, they 
are recorded only once for each cycle of five events. After the observer 
records the fifth statement in a cycle, he turns the form over and marks 
those alternatives to words 1-5 and 8 which best describe the context in 
which the five events described on the statement side took place. 




Continuing the same example, we mark ’’l.SCI” to indicate that the 
teacher is giving a science lesson; ”2 .SHOW” and ”2. TELL” (in Box T) to 
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indicate that the teacher (T) is demonstrating and explaining; "3. MID," 
which woiald indicate that the child is in the middle of the group; "4. ALL” 

(in Box. Ad) to indicate that the group in question contains all of the 
pupils (and an adult); and "8.ASSIGNED SEAT" (in Box L) to indicate that 
the pupil is in his regular seat. 

If one were to read this record without having seen the event, he would 
note that the child was quietly, sitting in the middle of the class in his 
own seat while the teacher was demonstrating something about science . 

While this does not describe exactly what happened, it describes the 
essential nature of the event in a way analogous to (thou^ less detailed 
than) an ecological record. 

When an observer visits a class he has with him one PROSE form for each 
pupil in the class, arranged in random order. He completes one cycle of 
five statements on each papil in turn, then another set of cycles, and 
finally a third (if time permits). In the longitudinal study we are 
planning, 100 observations of each child will be made, comprising 1,000 
statements describing samples of his behavior observed on 100 different 
days throiaghout the five years of the study. 

The number of different statements that can be made is very large. Over 
200,000 different sensible statements can be made up using only the 11 words 
on the statement side. Since each statement can also be modified accord- 
ing to the static conditions under which the event occurs, we mi^t say 
that the instrument has over a million items on it. Or at least, that it 
is possible to record much more detail in a PROSE record than could be used 
in any normal statistical analysis. It is in this wealth of detail that a 
PROSE record resembles an ecological one. The difference is, that with the 
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aid of SCRIBE, the high-speed test-scoring machine at ETS, all of this 
information may "be read into a conputer without further human intervention. 
Ultimately, as I have mentioned, the ccmiputer will have access to a sort of 
narrative description, 1,000 statements long, of the five-year school 
experience of each child in the study. 

The PROSE records on each child will he accompanied hy a vast amount 
of other infojmation about him — environmental, developmental, physical, 
social— to which his experiences in school may be related. As a hypothesis 
is developed, about such a relationship, it should be possible to find in the 
pupil’s PROSE records whatever information about his school experiences is 
relevant to it. It would then be a straightforward problem in computer 
programming to extract that information in quantitative form and perform 
whatever statistical analyses are needed. But that is another problem to 
be dealt with another day. 
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Figure 1. Statement Side of PROSE Form 
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Figure 2. Context Side of PROSE Form 



